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This paper reviews selected literature on 
education/business partnerships (E3Pb) , highlighting the status of 
partnership evaluations and current methodologies. Research by S. 
Otterbourg and D. Adams (1989), which surveyed about 24 EBPs to 
ascertain planning, implementation, and evaluation priorities, showed 
that only 25% of the programs used outcomes data to measure 
effectiveness. Otterbourg and Adams concluded that: the current state 
of the art of partnership program evaluation is at an elementary 
level; leaders are just beginning to use evaluation to collect data 
needed for maXing informed program- related decisions; and there is a 
laclc of precedent and experience and an absence of appropriate 
systems and instruments for evaluating partnership programs. Methods 
used to evaluate several EBPs are reviewed, including: documentation 
of collaUxration and long-rknge planning processes of the METRCLINK 
program by the Institute for Educational Leadership (lEL); 
qualitative review by Public/Private ventures of nine school/business 
partnerships t rough site visits, observations, interviews, and 
document review of program materials; site visits, observations, 
interviews, debrief ings with and written reports by technical 
assistants, questionnaire administration, phone follow-ups, and 
document review of the Ford Foundation's Url'an School/Community 
Dropout Prevention Collausoratives from 1986 to 1990; a Rand 
Corporation study of school/community collaborations via case 
studies, semi-structured interviews, and document searches; case 
studies and "mini" case reviews of EBPs; document review, phone 
interviews, and sits visits of 16 url:an colleges by the National 
Association of State Universities and Land-Grant Colleges in 1981-84; 
interviews and impact evaluations conducted by the Career Beginnings 
program; questionnaire administration, phone interview^, and site 
visits by the lEL to evaluate 70001 's Wor)c, Achievement, and values 
in Education Program in 1990; and day visits, interviews, and 
observations used in a study of local science education alliances in 
1988. The review illustrates the need for more systematic evaluation 
of EBP programs; and the need for additional inquiry, elaboration, 
^ amd refinement of evaluation models. (RLC) 
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Preface 



The U.S. Department of Education's Office of Educational Research 
and Improvement (OERI) funded the first cycle of four year 
projects under its new Educational Partnerships Program in 
September 1990. The Educational Partnerships Program is 
authorized by the Educational Partnerships Act of 1988, Title VT, 
subtitle A, Chapter 5 of the Omnibus Trade and Competitiveness 
Act of 1988 (Pub. L. 100-418) (20 U.S.C. 5031-5039). This 
legislation also requires the Secretary to conduct an annual 
evaluation of the grants made under the program and to 
disseminate information relating to the activities assisted. 

In performing the evaluation and dissemination responsibility, 
OERI has attempted to design its activities to build on what 
already is known, and to expand knowledge about the 
characteristics of effective partnerships and how to establish, 
sustain, and evaluate them. In order to establish a baseline for 
future work,, assess what already is known about educational 
partnerships, and review potential evaluation design options, 
OERI commissioned the following analyses and syntheses: 

Danzberger, Jacqueline P., "EDUCATIONAL PARTNERSHIPS 
PROGRAM: ANALYSIS OF PROJECT CHARACTERISTICS," 
December, 1990. 

Education Resources Group, "AN OVERVIEW OF EVALUATION 
RESEARCH ON SELECTED EDUCATIONAL PARTNERSHIPS," 
January, 1991. 

Education Resources Group, Inc. "OPTIONS FOR EVALUATING THE 
EDUCATIONAL PARTNERSHIPS PROGRAM," January, 1991. 

Grobe, Terry aad Susan P. Curnan and Alan Melchior, 
"SYNTHESIS OF EXISTING KNOWLEDGE AND PRACTICE IN THE FIELD 
OF EDUCATIONAL PARTNERSHIPS," December, 1990. 

This is one of the four commissioned reports. All four are 
available through ERIC. 



STATUS OF PARTNERSHIP EVALUATIONS 



A review of selected literature on educational partnerships 
reveals 1) the need for more systematic evaluation of such 
programs and 2) approaches or methodologies being used to 
evaluate them. The following types of assertions have recently 
been made. 



The research literature on school /business 
interactions is now huge; descriptive histories abound, 
coupled with informative "how-to" books illustrating 
specific practices and policies that support success. 
Most school and business participants hd^ve focused 
their attention on process, rather than product. While 
this focus is critical at the beginning, the 
development of relevant outcome measures is key to 
continuation and replication. Clearly, improving and 
expanding these interactions through rigorous 
evaluation procedures is the next step. 

- Roberta Trachtman, in, partnerships in 
Education: Measuring Their Success. 1989 



Few short-term partnership efforts have boen 
rigorously evaluated. While thoughtful analysos of 
issues are evident in selected research studies, these 
analyses confirm that qualitative information about 
metropolitan collaboration is scarce. Most information 
about public-private partnerships consists of 
quantitative data and anecdotal descriptions. 

- Institute for Educational Leadership, 1986 



My major concern... is that we have no means by 
which we can assess whether or not such col laborat Ives 
and activities. . .really have any Impact upon the goals. 
[We need) a reporting mechanism annually or bi annually 
as to whether or not the strategies are making any 
difference. 

- Oohn W. Porter, General Superintendent of 
Schools in Detroit, 1990 



The bottom line questions among results-oriented 
people is, what difference have school-business 
partnerships made? The authors have assembled some 
evidence and put forward their own answers... Beyond 
that, [they] argue for more attention to outcomes and 
they sketch the evaluation tasks that will be helpful 
in that. 

- Dale Mann, in Partnerships in Education; 
Measuring Their Success ^ 1989 

Susan Otterbourg and Don Adams, the authors referred to by 
Mann, surveyed some two dozen school-business partnerships around 
the country to ascertain planning, implementation, and evaluation 
priorities (1989). Only a quarter of the programs reported using 
outcomes data to measure effectiveness; they were keeping student 
records of courses passed, grades, results of achievement and 
content-based tests, attendance or dropout/ retention rates, and 
in one case, job performance ratings. About 20% were planning 
for outcomes evaluation in the future: these and the remaining 
55% were primarily using questionnaires and surveys to elicit 
feedback from participants. 

A review of program information included in their report 
indicates some sophisticated recordkeeping, process 
documentation, and understanding of which student outcomes might 
be measured (e.g., attendance and incidence of substance abuse). 
Despite this, Otterbourg and Adams concluded that "the current 
•state of the art» of partnership program evaluation is at an 
elementary level. • .Leaders are just beginning to use evaluation 



to collect data necessary to make informed programmatic 
decisions." Although programs reported that recordkeeping was 
relatively easy and straightforward, demonstrating outcomes was 
very difficult. The authors believe that there is a "lack of 
precedent and experience" and an "absence of appropriate systems 
and instruments" for evaluating partnership programs. More 
vigorous partnership evaluations of both processes and outcomes 
must be designed and funded. A detailed, step-by-step guide for 
partnerships wishing to develop their own monitoring and record 
keeping systems has been developed by Otterbourg (1990). 

CURRENT METHODOLOGIES 

Summaries of methods used to evaluate a number of school- 
business, school-community, and school-university partnerships 
are provided here. They reinforce, in general, that outcomes 
evaluations have not been conducted on many of these 
partnerships. Also, projects selected for this paper represent 
only a small sample of the studies being conducted on educational 
partnerships. A list of additional partnerships with 
evaluations, about which the reader might want to obtain 
information, is included in the Appendix. 

The Institute for Educational Leadership's (lEL) conclusion 
about a lack of "rigorous" evaluation led staff to develop and 




document the METROLINK program in 1984 and 1985. Eight cities 
participated in this demonstration of communityvide 
collaborations to develop human resources through education, 
employment, and training. lEL documented the processes of 
collaboration and long-range planning through program documents, 
interim project reports from the eight sites, correspondence, 
meeting minutes, and press coverage. They also gathered data 
from the site coordinators' exchange of information at quarterly 
group meetings and their interviews with local collaborative 
leaders. IEL*s report on METROLINK, prepared for the Danforth 
Foundation which funded the program, outlines five "stages" of 
collaborative development and twelve "themes" which apparently 
characterize collaborative efforts. 

In a major assessment of nine school/business partnerships, 
Public/Private Ventures (P/PV) attempted to 1) determine their 
significance for at--risk youth, 2) delineate their "concrete 
effects" on these students, and 3) predict their potential to 
improve public education (P/PV, 1987). Their study was 
qualitative: they conducted site visits, observations, 
interviews, and document review of program materials. 
Descriptive and quantitative information were collected when 
available. They analyzed student outcomes data for two of the 
nine projects • 

Results of these assessments are included in a two-volume 



set which analyzes school /business partnership components 
represented by the nine programs and presents ''extensive 
profiles" of them* P/PV concluded that partnerships "enabled 
schools to better serve at-risk youth" by providing increased 
access to employment/work experience, job coaching and 
preparation, and increased personal attention. 

A more recent, multi-year documentation of the Ford 
Foundation's Urban School-Community Dropout Prevention 
Collaboratives (1986-1990) studied the processes of 
collaboration, planning, technical assistance, generating public 
awareness, and local information gathering and evaluation 
efforts. A variety of methods were used to study these, 
including site visits, observations, interviews, debriefings with 
and written reports by technical assistants, a questionnaire, 
phone follow-ups, and document review (Clark, 1988^). 
Documenters used a set of structured reporting forms so that data 
could be collected in a standardized way across sites and the key 
processes of interest were studied at each data collection point. 

Through systematic content analysis of these reports and 
related program documents, Education Resources Group (ERG) , which 
is coordinating the collaboratives documentation, has generated a 
number of findings about collaboration between schools and 
communities. For example, ERG told OERI's Urban Superintendents 
Network about the need for "visible, facilitating leadership from 



the superintendent^" and the various ways this has been 
manifested in the better functioning dropout collaboratives 
(Clark, 1988^) • A full analysis of findings from this 
documentation will be presented in a manuscript being prepared 
with funds from The Ford Foundation and The Carnegie Corporation. 

Another broad study of school system-community 
collaborations, this one by the RAND Corporation, focused on 
improving school systems and the **specific actors" who played 
critical roles in that improvemant (Hill, et al, 1989). Rand 
selected six urban school systems and did case studies of the 
processes that affected school system improvement. Although 
their focus was on the educational systems, they saw the role of 
business and community leaders as essential to improvement. 
During 1987-88, RAND field teams did a series of semi -structured 
interviews and document searches during two to four visits per 
city. The interviews — dozens in each site including those with 
business, civic, and government leaders — provided the basis for 
their case studies. 

Seven "case studies" and 22 "mini" cases of business-school 
partnerships — including the Boston Compact and the California 
Roundtable — are included in the 1988 Co:imnittee for Economic 
Development publication American Business and the Public Schools 
(Levine and Trachtman) . Authors of the case studies (many of 
whom were involved in implementation of the partnership) offered 



mixed assessments of program impact (i.e., pointed out successes 
but also cited problems in outcomes data) and raised a number of 
questions and caveats about outcomes of those partnerships. 

Researchers for the National Associatic of State 
Universities and Land-Grant Colleges reported on a 1981-84 
project in which 16 urban colleges committed financial or other 
resources toward improving opportunities for urban youth through 
partnerships with public schools (Mocker, et al, 1988). The 
authors also studied the collaborative process, by seeking common 
factors that appeared to contribute to the success of the most 
"productive" projects. They used document review, phone 
interviews, and site visits. It is not clear how thej^^^ef^ined 
productive collaboration, but they report six "lessons" that 
emerged from their study, which indicate the conditions under 
which productive collaborations operate. 

Another college-school partnership program, which also uses 
business resources, is the CAREER BEGINNINGS program (Bloomfield, 
1989). Its purpoL-- is to increase the likelihood that minority 
and disadvantaged high school graduates enter college or get good 
jobs. Administered out of Brandeis University since 1985, the 
program started with 24 two and four-year colleges around the 
country. The program is cited in this revi^sw because several 
evaluative studies have been conducted on it. These and the 
outcomes they focus on are described below. 
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Manpower Demonstration Research Corporation (MDRC) conducted 
first-year interviews of CAREER BEGINNINGS participants, and 
wro:e a preliminary "impact" evaluation focusing on such student 
outcomes as high school graduation rates and college attendance 
and retention rates. They did a "final" report after the second 
program year (1987), which included data on a control group. An 
unpublished Louis Harris and Associates survey on mentoring was 
conducted in 1989 with Commonwealth Fund support. A Bos. ton-based 
group, Education Matters, conducted an imple.nentation analysis 
funded by The MacArthur Foundation: that report, which will be 
out soon, focuses on administration, staffing, program 
activities, and level of partnership activities. Site program 
administrators are keeping data tracking students through high 
school and college; although there are no comparison group data, 
site statistics are being contrasted with the U.S. Department of 
Education's data on college-going rates for these types of 
students. (Information about evaluation of CAREER BEGINNINGS was 
provided by William Bloomfield via telephone on 11/9/90) . 

The Institute for Educational Leadership used different data 
collection methods than they had for METROLINK to evaluate 
70001 's Work, Achievement, and Values in Education (WAVE) program 
in 1990. In its first implementation year at the time, WAVE was 
attempting to develop community - high school linkages to improve 
at-risk students' prospects for academic and personal success. 
After distributing questionnaires and conducting phone interviews 
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with WAVE teachers, site administrators; and 70001 program 
specialists, lEL staff visited 13 of 52 WAVE sites and 
administered in-person, structured interviews with students, 
teachers, administrators, staff, parents, and community members 
involved in WAVE and non-WAVE teachers. lEL concluded in its 
final report (1990) that WAVE was having positive effects on 
students and teachers, but pointed out that "long-term results 
will need to be examined" and "continual academic progress needs 
longer term validation" through test scores and other outcomes 
data. 

A study of local science education alliances evolved from an 
evaluation of 20 collaboratives funded by The Carnegie 
Corporation to a "careful look" at new, inter-institutional 
approaches to the improvement of science education around the 
country. Researchers Myron and Ann Atkin conducted two to three 
day visits in 1988 to 30 par Iznerships "with strong reputations 
for quality of inter-institutional cooperation." They collected 
qualitative information through interviews and observations. 
Their findings are discussed in a book which also includes rich 
descriptions of a variety of science education alliances and 
their sources of support (Atkin and Atkin, 1989) . 

DEVELOPING MODELS 

Shirley Hord of the University of Texas synthesized the 
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th«)oretical literature which attempts to distinguish between 
••collaboration" and ••cooperation** among partners (1986). In that 
context she discussed several studies which focused on 
••successful •• collaborations, but did not explain how these 
conclusions were derived. Her final statement continues to be 
illustrative of the state-of-art of evaluating partnerships. 



A great deal of additional inquiry, elaboration, 
and refinement is necessary. Research is needed that 
undertakes comparative studies of the various ways of 
interrelating: cooperation, coordination ^ 
collaboration, and so on. With a further delineation 
of the components of each process, including an 
explication of the costs and benefits of each, perhaps 
we would learn the requirements of each method, the 
J imitations, and the effectiveness of each type under 
specific circumstances. 
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Appendix 



ni-hft,' seleci-Art Partners ^^p« with Evaluation Studies * 

College Bound, GE Foundation, Fairfield, CT. 

Education for une Future Project, c/o Managing Excellence in 
Education, Irvine, CA. 

Educational EQuality Project, The College Board, New York City. 

Generations Together, The University of Pittsburgh, PA. 

New Futures Initiative, Annie E. Casey Foundation, Greenwich, C 

Panasonic Foundation School Improvement Program, Secaucus, NJ. 

Syracuse University Project Advance, Syracuse, NY. 

urban Mathematics Collaboratives, c/o Education Development 
Center, Newton, MA. 



* Also see Otterbourg and Adams, Partn^rshlpg in E«aug^tiQ»li for 
references to two dozen partnerships. 



